Video processing device, video display device and video processing method therefor and program thereof

ABSTRACT

In the video processing device, a correction amount update determining unit determines whether to update a correction amount based on an input image obtained from an image input unit and when a cut point is detected or when the number of frames in a number of frames storing unit exceeds a fixed value, determines that update is required, and at this time, a correction amount obtaining unit obtains a new correction amount based on the input image. Then, if no cut point is detected, an amount of change of a correction amount in time is limited and the obtained result is recorded in a correction amount storing unit, and an image correcting unit conducts quality improving correction processing with respect to the input image based on a correction amount recorded in the correction amount storing unit and the image corrected is sent to an image output unit.

BACKGROUND OF THE INVENTION

1. Field of the Invention

The present invention relates to a video processing device, a videodisplay device and a video processing method therefor, and a programthereof and, more particularly, to a method of automatically improvingquality of a moving image.

2. Description of the Related Art

Image quality improving represents subjecting an original image to imagecorrection processing so as to make still picture and moving pictureclearer. Among correction processing for improving image quality aresaturation correction and γ (gamma) correction.

Saturation correction is correction intended to adjust saturationindicative of a density of color. Since people are apt to prefer animage whose saturation is high, saturation correction is often conductedso as to adjust saturation of an original image to be high. γ correctionis correction intended to adjust brightness of an image. People preferimages of appropriate brightness to images too dark or too bright.Adjusting such brightness is γ correction.

Other than those mentioned above, various kinds of corrections exist andusing these correction processing methods to make an image clearer isrepresented as image quality improving processing. As theabove-described image quality improving methods, there conventionallyexist such methods as set forth below.

For improving quality of a still image, various still picture qualityautomatic improving techniques have been used. Still picture qualityimproving techniques here include those recited in “Color Image QualityAutomatic Improvement by Adjustment of Saturation, Contrast, andSharpness” (Inoue and Tajima, the 24th Image Technology ConferenceProceedings 3-3, 1993)(Literature 1), Japanese Patent Laying-Open(Kokai) No. Heisei 09-147098 (Literature 2), Japanese Patent Laying-Open(Kokai) No. Heisei 10-150566 (Literature 3) and “Automatic ColorCorrection Method Realizing Preferable Color Reproduction” (Tsukada,Funayama, Tajima, Color Forum JAPAN 2000 Proceedings, pp. 9-12, 2000)(Literature 4).

In the automatic image quality improving methods recited in theseliteratures, a certain feature amount is extracted from an input imagecomposed of still images and a correction amount is determined based onthe feature amount to conduct correction for quality improving. Featureamount here represents, for example, an average luminance of a darkregion within a screen or an average tone value of each of RGB (R: red,G: green, B: blue) in a bright region within a screen.

In the following, one example of each correction method will bedetailed. One example of saturation correction realization methods isshown in FIG. 28. In the present saturation correction realizationmethod, first create a histogram of S values using an HSV (HueSaturation Value) coordinate system or the like with respect to an inputimage illustrated in FIG. 28( a) [see FIG. 28( b)]. The HSV coordinatesystem here is recited in “Color Gamut Transformation Pairs” (A. R.Smith, Computer Graphics, vol. 12, pp. 12-19, 1978).

An S value in the HSV coordinate system denotes saturation, and ahistogram of S values therefore can be considered as a histogram ofsaturation. In the histogram here generated, assume that a highsaturation portion where an area ratio to the total number of pixels hasa fixed rate “a” is a high saturation region. Then, calculate an averagesaturation SAF of the high saturation region [see FIG. 28( b)].Calculate a correction amount Copt from the average saturation SAFaccording to the following expression:Copt=SAFopt/SAF  (1)

where the average saturation SAFopt represents an optimum value that asaturation image of an input image can take.

The larger thus calculated correction amount Copt becomes, the moresaturation will be highlighted [see FIG. 28( c)]. The value of c0 in thefigure is that obtained when a range of the saturation S of an inputimage is expanded to the largest and when c=c0, the saturation S of theinput image extends to the largest range as shown in FIG. 28( c) [seeFIG. 28( d)].

In image quality improving, obtain an S value from an RGB value of eachpixel of a frame image and linearly transform the obtained valueaccording to the following expression:S′=C0pt×S  (2).

After the transformation, restoring the value to an RGB value againleads to completion of an image being corrected. The above-describedsaturation correction is recited in Literature 1.

One example of exposure correction realization methods is shown in FIG.29. In the present exposure correction realization method, first createa histogram of Y values using an XYZ coordinate system with respect toan input image illustrated in FIG. 29( a) [see FIG. 29( b)]. Since a Yvalue denotes luminance, the histogram of Y values can be considered asa luminance histogram.

At this time, with all times the number of pixels set to be m, and witha value of the m-th highest luminance as Zmax and a value of the m-thlowest luminance as Zmin, obtain an intermediate value M of thehistogram according to the following expression:M=(Zmax+Zmin)/2  (3).

A γ value with which the intermediate value M becomes M0, half thedynamic range, after the transformation can be obtained by the followingexpression:γ=[log(255/M0)]/[log(255/M)]  (4).

Exposure correction is realized by first obtaining a Y value from an RGBvalue of each pixel of a frame image and subjecting the input image togamma correction by using a γ value obtained by the expression (4) andthe following expression with respect to the obtained Y value [see FIGS.29( c) and (d)].

$\begin{matrix}{Y = {\frac{255}{255^{\gamma}}Y^{\gamma}}} & (5)\end{matrix}$

As to the above-described exposure correction, recitation is found inLiterature 3.

One example of white balance correction realization methods is shown inFIG. 30. In the present white balance correction method, first create aluminance histogram with respect to an input image illustrated in FIG.30( a) using an XYZ coordinate system or the like [see FIG. 30( b)].

At this time, with “a” times the number of pixels set to be m, considera mean value of the respective tone values of pixels having the highestto the m-th highest luminances as a white point of the image. With thewhite point RGB value denoted as (wr, wg, wb) and a white color RGBvalue obtained after adjustment as (wr0, Wg0, wb0), obtain white balancecorrection amounts r, g, b according to the following expressions:r=wr0/wrg=wg0/wgb=wb0/wb  (6).

Based on the above obtained correction amounts and the followingexpressions, linear transformation of each tone value will realize whitebalance correction as illustrated in FIG. 30( c):R′=r×RG′=g×GB′=b×B  (7).As to the above-described white balance correction, recitation is foundin Literature 2.

One example of contrast correction realization methods is shown in FIG.31. In the contrast correction realization method, first, create ahistogram of Y values, that is, a luminance histogram, with respect toan input image illustrated in FIG. 31( a) using an XYZ coordinate systemor the like [see FIG. 31( b)].

At this time, with “a” times the number of pixels set to be m, obtain anaverage luminance Vmax of pixels having the highest to the m-th highestluminances. Similarly, obtain an average luminance Vmin of pixels havingthe lowest to the m-th lowest luminances [see FIG. 31( b)].

Based on these values, obtain the following expression which is astraight line passing the coordinates (Vmin, 0), (Vmax, 255):V′=a×V+b  (8).

In the expression, V denotes a luminance Y value of a pixel of anoriginal image and V′ denotes a Y value of the pixel transformed. Bylinearly transforming a luminance of each pixel using the expression (8)for the inversion into an RGB value realizes contrast highlighting. Asto the above-described contrast correction, recitation is found inLiterature 1.

One example of sharpness correction realization methods is shown in FIG.32. In the present sharpness correction method, first subject an inputimage shown in FIG. 32( a) to a high-pass filter to extract an edgecomponent as illustrated in FIG. 32( b). With ss representing ahigh-pass filter, E(V) representing an edge region, AE(V) representingan area of an edge region, V representing a luminance and ES0ptrepresenting an optimum sharpness of the image in question, thesharpness correction amount k will be obtained by the followingexpression:

$\begin{matrix}{k = \frac{{{ES}_{opi} \cdot {A_{E}\left( V^{\prime} \right)}} - {\int{\int_{E{(V^{\prime})}}{{{V \otimes {ss}}}{\mathbb{d}x}{\mathbb{d}y}}}}}{\int{\int_{E{(V^{\prime})}}{{{V \otimes {ss} \otimes {ss}}}{\mathbb{d}x}{\mathbb{d}y}}}}} & (9)\end{matrix}$

Using k obtained by the expression (9), the sharpening will be conducedbased on the following expression:V′=V+k(V{circle around (x)}ss)  (10)

By inversely transforming an RGB value from V′ obtained by theexpression (10), sharpness correction will be realized. As to theabove-described sharpness correction, recitation is found in Literature1.

One example of preferable color correction realization methods is shownin FIG. 33. Preferable color correction is making look of color of animage (representing how the color is perceived by a person, which isalso the case in the following description) be more preferable to humaneyes by approximating the color of the image to color of the objectremembered by a person. Specific processing shown in FIG. 33 as anexample is conducted in a manner as follows.

Calculate a hue of each pixel of a frame image shown in FIG. 33( a) tocreate such a histogram of hues as illustrated in FIG. 33( b). Correctthe histogram to make hues related to skin color, color of the sky andgreen color of plants be those producing more preferable colors byadapting color correction parameters given in advance according to eachdivisional hue region as shown in FIG. 33( c).

As a result, as illustrated in FIG. 33( d), the image has morepreferable colors with only the colors of skin, the sky and green ofplants changed. Subjecting the image to such processing realizespreferable color correction. The preferable color correction is intendedfor obtaining color that one finds preferable when looking only at acorrected image and is conducted based on the contents of know-howaccumulated in a data base for a long period of time. As to theabove-described preferable color correction, recitation is found inLiterature 4.

Using such still picture automatic quality improving techniques asmentioned above realizes quality improving of still picture. Used forimproving quality of a moving image is a method of improving qualityusing a fixed parameter. Fixed parameter is a correction amountparameter fixed to a constant value in order to conduct certaincorrection for a moving image. Fixed parameters are, for example, asfollows.

As shown in FIG. 34, generate images corrected with γ values as variousgamma correction parameters in the expression (5) changed and comparethe images to obtain an optimum γ value with which the image looksclearer by a subjective evaluation test. At the time of subjecting amoving image to γ correction, when the correction is conducted using anoptimum γ value without changing a γ value, the γ value can beconsidered as a fixed parameter. Technique for improving quality ofimages using such fixed parameter not only in γ correction but also invarious correction processing is employed in moving image qualityimproving processing.

With the above-described conventional systems in which a correctionamount is given by a fixed parameter, however, the moving image qualityimproving techniques fail to appropriately change a correction amountaccording to a video source and video shooting conditions.

Moving image has its image quality largely varying depending on itsvideo source and video shooting conditions. In terms of the differencein video sources, a moving image obtained from a DVD (Digital VersatileDisc) deck has high saturation and relatively high contrast, while amoving image shot by an individual person using a home-use digital videocamera or the like has low saturation and low contrast as well becauseof properties of cameras.

In terms of shooting conditions, scenery shot by a digital video camerain cloudy weather and that shot in fine weather will partially differ insaturation and contrast. Quality of a moving image thus varies largelydepending on circumstances.

On the other hand, with a correction amount determined by a fixedparameter, while an image taken by a digital video camera is clear, aDVD image might exhibit unnatural look because of too much correction insome cases. Although such a case can be coped with by a method ofobtaining a fixed parameter for each video source and manually switchingand using the parameters according to each video source, the method isnot convenient because it fails to cope with different image qualitiescaused by different shooting conditions and needs manual switching.

SUMMARY OF THE INVENTION

Under these circumstances, an object of the present invention is toeliminate the above-described shortcomings and provide a videoprocessing device, a video display device and a video processing methodtherefor and a program thereof which enable a correction amount to beappropriately changed according to image quality of an input movingimage and enable quality of a moving image to be automatically improved.

According to the first aspect of the invention, a video processingdevice comprises

correction amount obtaining means for obtaining a correction amount fromsequentially applied moving images, and

image correcting means for subjecting an input moving image to qualityimproving correction processing based on a correction amount obtained bythe correction amount obtaining means.

In the preferred construction, the video processing device furthercomprises image input means for obtaining a frame image fromsequentially applied moving images and outputting the image to thecorrection amount obtaining means, wherein

the correction amount obtaining means obtains the correction amount fromthe frame image and the image correcting means subjects the frame imageto quality improving correction processing based on the correctionamount.

In another preferred construction, the video processing device furthercomprises correction region cutting-out means for, before the movingimage is subjected to correction processing, cutting out a region to becorrected from the moving image in question, and

image composing means for combining a region to be corrected which iscut out by the correction region cutting-out means and a region not tobe corrected which is a remainder left after the region to be correctedin question is cut out.

In another preferred construction, the video processing device furthercomprises correction amount change limiting means for limiting an amountof change between a correction amount of a current frame image obtainedby the correction amount obtaining means and a stored correction amountof a preceding frame image.

In another preferred construction, the video processing device furthercomprises lapse of fixed time detecting means for counting the number offrames from a frame image whose the correction amount is lastly updatedto a current frame image to determine whether the number of framesexceeds a fixed value, and

correction amount update determining means for giving an instruction toupdate the correction amount when the lapse of fixed time detectingmeans determines that a fixed time has elapsed.

In another preferred construction, the video processing device furthercomprises cut point detecting means for detecting a cut point indicativeof switching of a scene in the moving image based on a change of afeature amount obtained from each frame image, and correction amountupdate determining means for giving an instruction to update thecorrection amount when the cut point detecting means detects the cutpoint.

In another preferred construction, the video processing device furthercomprises lapse of fixed time detecting means for counting the number offrames from a frame image whose the correction amount is lastly updatedto a current frame image to determine whether the number of framesexceeds a fixed value,

cut point detecting means for detecting a cut point indicative ofswitching of a scene in the moving image based on a change of a featureamount obtained from each frame image, and

correction amount update determining means for giving an instruction toupdate the correction amount either when detection of a lapse of a fixedtime is made by the lapse of fixed time detecting means or whendetection of the cut point is made by the cut point detecting means.

In another preferred construction, the correction amount obtaining meansincludes

correction amount calculating means for calculating n (n≧1) kinds ofarbitrary correction amounts, and

the image correcting means includes

n kinds (n≧1) of arbitrary correcting means.

In another preferred construction, the correction amount obtaining meansincludes at least one of

white balance correction amount calculating means for calculating awhite balance correction amount of the moving image,

contrast correction amount calculating means for calculating a contrastcorrection amount of the moving image,

saturation correction amount calculating means for calculating asaturation correction amount of the moving image,

exposure correction amount calculating means for calculating an exposurecorrection amount of the moving image,

sharpness correction amount calculating means for calculating asharpness correction amount of the moving image, and

preferable color correction amount calculating means for calculating apreferable color correction amount indicative of a correction amountrequired for a preset preferable color in the moving image, and

the image correcting means includes at least one of

white balance correcting means for conducting white balance correctionof the moving image corresponding to the correction amount obtainingmeans,

contrast correcting means for conducting contrast correction of themoving image,

saturation correcting means for conducting saturation correction of themoving image,

exposure correcting means for conducting exposure correction of themoving image,

sharpness correcting means for conducting sharpness correction of themoving image, and

preferable color correcting means for conducting the preferable colorcorrection of the moving image.

In another preferred construction, the image correcting means conducts

correction with respect to a moving image corrected by the imagecorrecting means at a preceding stage based on a correction amountcalculated by the correction amount obtaining means and the correctionamount obtaining means calculates the correction amount of a movingimage corrected by the image correcting means corresponding to thecorrection amount obtaining means at a preceding stage.

In another preferred construction, the correction amount obtaining meansincludes

an evaluation region cutting-out means for cutting out an evaluationimage region for calculating a correction amount from the frame image.

In another preferred construction, the correction amount obtaining meansincludes upper limit value adjusting means for comparing a correctionamount obtained in advance and an upper limit value and when the valueis larger than the upper limit value, replacing the value with apredetermined set value.

In another preferred construction, the correction amount change limitingmeans includes

change amount calculating means for calculating an amount of changebetween a latest correction amount and a correction amount of apreceding frame and change amount limiting means for limiting an amountof change of the correction amount based on a maximum change range.

In another preferred construction, the cut point detecting means isstructured to consider a result of comparison of a color histogramgenerated based on color information of each pixel of the moving imagewhich is conducted on a frame basis as a feature amount and detect a cutpoint of the moving image based on a change of the feature amount.

In another preferred construction, the cut point detecting means isstructured to, at the time of generating the color histogram from themoving image, generate the color histogram after thinning out the imageat fixed intervals.

According to the second aspect of the invention, a video processingdevice comprises

image input means for obtaining a frame image from sequentially appliedmoving images, and

cut point detecting means for detecting a cut point indicative ofswitching of a scene in the moving image based on a change of a featureamount obtained from each frame image.

In the preferred construction, the cut point detecting means isstructured to consider a result of comparison of a color histogramgenerated based on color information of each pixel of the moving imagewhich is conducted on a frame basis as a feature amount and detect a cutpoint of the moving image based on a change of the feature amount.

In another preferred construction, the cut point detecting means isstructured to, at the time of generating the color histogram from themoving image, generate the color histogram after thinning out the imageat fixed intervals.

According to the third aspect of the invention, a video display devicecomprises

moving image correction amount obtaining means for obtaining N (N≧1)kinds of correction amounts from sequentially applied moving images,

image correcting means for conducting N (N≧1) kinds of quality improvingcorrections with respect to the moving image based on a correctionamount obtained by the moving image correction amount obtaining means,and

image display means for displaying a moving image corrected by the imagecorrecting means.

According to another aspect of the invention, a video processing methodcomprising the steps of

obtaining a correction amount from sequentially applied moving images,and

conducting quality improving correction with respect to the appliedmoving image based on the obtained correction amount.

In the preferred construction, the video processing method comprisingthe steps of

obtaining a correction amount from each frame image forming sequentiallyapplied moving images, and

conducting quality improving correction with respect to the frame imagebased on the obtained correction amount.

In another preferred construction, the video processing methodcomprising the steps of

updating a correction amount for every N frames (N≧1).

In another preferred construction, the video processing methodcomprising the steps of

checking the input moving image on a frame basis and when a cut pointindicative of switching of a scene in the input moving image isdetected, updating a correction amount.

In another preferred construction, the video processing methodcomprising the steps of

updating a correction amount for every N frames (N≧1), and

checking the input moving image on a frame basis and when a cut pointindicative of switching of a scene in the input moving image isdetected, updating a correction amount.

In another preferred construction, the video processing methodcomprising the steps of

obtaining n (n≧1) kinds of arbitrary correction amounts at the time ofobtaining the correction amount from the input moving image, and

conducting n (N≧1) kinds of arbitrary quality improving corrections withrespect to the input moving image based on the obtained correctionamount.

In another preferred construction, the correction amount obtaining stepincludes at least one of the steps of

calculating a white balance correction amount of the moving image,calculating a contrast correction amount of the moving image,calculating a saturation correction amount of the moving image,calculating an exposure correction amount of the moving image,calculating a sharpness correction amount of the moving image, andcalculating a preferable color correction amount indicative of acorrection amount required for a preset preferable color in the movingimage, and

the quality improving correction conducting step includes at least oneof the steps of

conducting white balance correction of the moving image corresponding tothe correction amount obtaining step, conducting contrast correction ofthe moving image, conducting saturation correction of the moving image,conducting exposure correction of the moving image, conducting sharpnesscorrection of the moving image, and conducting the preferable colorcorrection of the moving image.

In another preferred construction, at the quality improving correctionconducting step, correction is conducted with respect to a moving imagecorrected at the step of conducting quality improving correction at apreceding stage based on a correction amount calculated at the step ofobtaining a correction amount, and

at the correction amount obtaining step, the correction amount iscalculated from a moving image corrected at the step of conductingquality improving correction corresponding to the step of obtaining acorrection amount at a preceding stage.

In another preferred construction, the video processing method furthercomprising the step of

limiting an amount of change between the obtained correction amount of acurrent frame and a stored correction amount of a preceding frame.

In another preferred construction, the video processing method furthercomprising the step of

cutting out an evaluation image region necessary for obtaining thecorrection amount from the frame image, and

obtaining the correction amount from the cut-out evaluation image.

In another preferred construction, the video processing method furthercomprising the step of

at the detection of the cut point, considering a result of comparison ofa color histogram generated based on color information of each pixel ofthe frame image which is conducted on a frame basis as a feature amountand detecting a cut point of the moving image based on a change of thefeature amount.

In another preferred construction, the video processing method furthercomprising the step of

when detecting the cut point, at the time of generating the colorhistogram from the frame image, generating the color histogram afterthinning out the image at fixed intervals.

In another preferred construction, the video processing method furthercomprising the step of

when a moving image partly flows on such a screen as a screen of apersonal computer, before subjecting the moving image to correctionprocessing, cutting out a region to be corrected from the moving imagein question,

subjecting the cut-out image to be corrected to image correction, and

combining the region to be corrected which is subjected to imagecorrection and a region not to be corrected which is a remainder leftafter the region to be corrected in question is cut out to output thecombined image.

According to a further aspect of the invention, a video processingmethod comprising the steps of

obtaining a frame image from sequentially applied moving images, and

detecting a cut point indicative of switching of a scene in the movingimage based on a change of a feature amount obtained from each frameimage.

In the preferred construction, at the cut point detecting step, a resultof comparison of a color histogram generated based on color informationof each pixel of the moving image which is conducted on a frame basis isconsidered as a feature amount and a cut point of the moving image isdetected based on a change of the feature amount.

In another preferred construction, at the cut point detecting step, atthe time of generating the color histogram from the moving image, thecolor histogram is generated after thinning out the image at fixedintervals.

According to a still further aspect of the invention, a video processingprogram for controlling a computer to execute video processing,comprising the functions of

obtaining at least one correction amount from moving images sequentiallyapplied to the computer,

comparing the obtained correction amount with a correction amountobtained from at least one of preceding past frames to suppress a changein correction amount, and

subjecting a frame image to quality improving correction based on thesuppressed correction amount.

As described in the foregoing, at the determination of a correctionamount of a latest frame, by obtaining an amount of change between acorrection amount of the latest frame and that of a past frame tominimize an amount of change to a degree that causes no flickering etc.,the present invention enables quality of a moving image to beautomatically improved without exhibiting uncomfortable look such asflickering.

Since according to the present invention, detection of a cut pointenables detection of switching of a scene, quality improving can beconducted with an appropriate correction amount according to variationof scenes.

When a scene in a moving image is switched, an image whose look isdifferent from that of the former image is applied to a system. Thus,when look of an image largely changes, an optimum correction amount foreach image might be changed. Because of having the correction amount asa fixed parameter, the conventional methods fail to conduct correctionof moving images with an appropriate correction amount.

Since upon detecting switching of a scene, the present invention isallowed to newly obtain an appropriate correction amount automaticallyby a moving image correction amount obtaining unit, it is possible toconduct correction for improving image quality with an appropriatecorrection amount for each different scene detected.

Because according to the present invention, when a cut point detectionand in-frame correction amount obtaining unit evaluates an image, anevaluation region cut-out unit cuts an evaluation region to have anarbitrary area, automatic image improving is possible irrespective ofdifference in video sources.

Moving image has a display region largely varying depending on its inputsource. In a case of a TV image or a game image, the image is displayedon the entire region of a TV monitor. On the other hand, in a case of ahi-vision image, films and the like, upper and lower black zones aredisplayed to reduce an image display region.

The image quality improving correction method recited as an example inthe present invention in some cases fails to obtain an appropriatecorrection amount due to the effect of the black zones. In contrastcorrection, for example, in which a correction amount is determinedbased on a dark region of a screen, when the whole of the screen is usedas an evaluation region, the correction amount will be determined basedon a region of upper and lower black zones to prevent optimum improvingof quality of images at other region than the black zones.

Also as to the cut point detection exemplified in the preset invention,a cut point might not be detected appropriately because of the effectsof the black zone. Using the evaluation region cut-out unit describedhere, however, solves these problems to enable appropriate cut-out of ascene and enable quality of a moving image to be improved with anappropriate correction amount.

Since according to the present invention, the video processing device isstructured including a saturation correction unit, an exposurecorrection unit, a white balance correction unit, a contrast correctionunit, a sharpness correction unit and a preferable color correction unitas still picture quality automatic improving techniques independently ofeach other in various combinations, various image quality improvingcorrections can be automatically conducted.

In addition, incorporation of not only the above-described six units butalso other image quality improving unit enables the present invention toimprove quality of a moving image in the same manner as the above sixunits do.

Since in cut point detection according to the present invention,thinned-out images are generated, cut point detection is possibleirrespective of interlace characteristics.

The present invention is premised on various kinds of images such as aTV image and a DVD image as an input image. Among these images, twoimages are in some cases seen overlapped with each other in one frame.This is a phenomenon occurring when a video signal as an interlace imagehas 30 frames/sec, while the main moving images has 24 frames/sec. Thedifference in frame rates causes such a phenomenon that two images areseen overlapped with each other in one frame as described above.

When two images thus overlap with each other, an image of a precedingscene and an image of a succeeding scene are seen overlapped with otherin one frame at a cut point. This results in increasing similaritybetween the preceding and the succeeding frames at the cut point tohinder cut point detection which has been described in relation to theabove effects in some cases. Therefore, thinning out images to eliminateoverlap of images enables cut point detection to be conducted moresatisfactorily.

According to the present invention, since a moving image region in aninput image can be cut out by a correction region cut-out unit at thetime of image quality improving correction processing and because animage composing unit is provided for restoring cut out moving imagessubjected to image quality improving correction to such arrangement asin an original computer screen, out of an image in which a moving imageflows at a part of a screen such as a computer screen, only the movingimage region can be improved to have high quality and displayed.

On a computer screen, when an application for displaying moving imagessuch as a media player is activated, there appears an image in whichstill picture forms a surrounding area and a moving image locally flows.The correction region cut-out unit cuts out such image into a stillpicture region and a moving image region. As a result, image qualityimproving correction can be made of the moving image region using anoptimum correction amount.

Thus, the corrected moving image is combined with the surrounding stillpicture region by the image composing unit to have a moving image whosequality is improved with a correction amount optimum for a moving image,while display of the computer screen remains original display.

Other objects, features and advantages of the present invention willbecome clear from the detailed description given herebelow.

BRIEF DESCRIPTION OF THE DRAWINGS

The present invention will be understood more fully from the detaileddescription given herebelow and from the accompanying drawings of thepreferred embodiment of the invention, which, however, should not betaken to be limitative to the invention, but are for explanation andunderstanding only.

In the drawings:

FIG. 1 is a block diagram showing a structure of a video processingdevice according to a first embodiment of the present invention;

FIG. 2 is a block diagram showing a detailed structure of a correctionamount obtaining unit illustrated in FIG. 1;

FIG. 3 is a diagram for use in explaining a method of adjusting acorrection amount by the correction amount obtaining unit illustrated inFIG. 1 using an upper limit value and a set value;

FIG. 4 is a block diagram showing an example of a detailed structure ofan image correcting unit illustrated in FIG. 1;

FIG. 5 is a flow chart showing operation of the video processing deviceaccording to the first embodiment of the present invention;

FIG. 6 is a flow chart showing operation of the correction amountobtaining unit illustrated in FIG. 1;

FIG. 7 is a flow chart showing operation of the correction amountobtaining unit illustrated in FIG. 1;

FIG. 8 is a block diagram showing a structure of a video processingdevice according to a second embodiment of the present invention;

FIG. 9 is a block diagram showing a detailed structure of a correctionamount change limiting unit illustrated in FIG. 8;

FIG. 10 is a diagram showing one example of processing conducted by thecorrection amount change limiting unit illustrated in FIG. 8;

FIG. 11 is a flow chart showing operation of the video processing deviceaccording to the second embodiment of the present invention;

FIG. 12 is a block diagram showing a structure of a video processingdevice according to a third embodiment of the present invention;

FIG. 13 is a block diagram showing a detailed structure of a correctionamount update determining unit illustrated in FIG. 11;

FIG. 14 is a block diagram showing a detailed structure of a cut pointdetecting unit illustrated in FIG. 13;

FIG. 15 is a diagram for use in explaining a color histogram for use inthe cut point detecting unit shown in FIG. 13;

FIG. 16 is a diagram showing a relationship between transition of adifference value and a threshold value in an example of feature pointcomparison for use in the cut point detecting unit of FIG. 13;

FIG. 17 is a flow chart showing operation of the video processing deviceaccording to the third embodiment of the present invention;

FIG. 18 is a flow chart showing operation of the correction amountupdate determining unit illustrated in FIG. 12;

FIG. 19 is a flow chart showing operation of the cut point detectingunit illustrated in FIG. 13;

FIG. 20 is a block diagram showing a structure of a video processingdevice according to a fourth embodiment of the present invention;

FIG. 21 is a schematic diagram showing processing of the videoprocessing device according to the fourth embodiment of the presentinvention;

FIG. 22 is a flow chart showing operation of the video processing deviceaccording to the fourth embodiment of the present invention;

FIG. 23 is a block diagram showing a structure of a video processingdevice according to a fifth embodiment of the present invention;

FIG. 24 is a block diagram showing a detailed structure of a correctionamount obtaining unit illustrated in FIG. 23;

FIG. 25 is a block diagram showing an example of a detailed structure ofan image correcting unit illustrated in FIG. 23;

FIG. 26 is a block diagram showing a structure of a video display deviceaccording to a sixth embodiment of the present invention;

FIG. 27 is a block diagram showing a structure of a video processingdevice according to a seventh embodiment of the present invention;

FIG. 28 is a diagram for use in explaining one example of conventionalsaturation automatic improving correction;

FIG. 29 is a diagram for use in explaining one example of conventionalexposure automatic improving correction;

FIG. 30 is a diagram for use in explaining one example of conventionalwhite balance automatic improving correction;

FIG. 31 is a diagram for use in explaining one example of conventionalcontrast automatic improving correction;

FIG. 32 is a diagram for use in explaining one example of conventionalsharpness automatic improving correction;

FIG. 33 is a diagram for use in explaining one example of conventionalpreferable color correction;

FIG. 34 is a diagram for use in explaining one example of a conventionalquality improving technique using a fixed parameter.

DESCRIPTION OF THE PREFERRED EMBODIMENT

The preferred embodiment of the present invention will be discussedhereinafter in detail with reference to the accompanying drawings. Inthe following description, numerous specific details are set forth inorder to provide a thorough understanding of the present invention. Itwill be obvious, however, to those skilled in the art that the presentinvention may be practiced without these specific details. In otherinstance, well-known structures are not shown in detail in order tounnecessary obscure the present invention. FIG. 1 is a block diagramshowing a structure of a video processing device according to a firstembodiment of the present invention. In FIG. 1, the video processingdevice according to the first embodiment of the present inventionincludes an image input unit 1 to be connected with a DVD (DigitalVersatile Disc) player, a computer, a game apparatus, a DV (DigitalVideo) camera, etc. to obtain a frame image forming an input movingimage, a data processing device 2 operative by program control, astorage device 3 for storing information and an image output unit 4 foroutputting a corrected frame image to the outside of the device.

The storage device 3 includes a correction amount storing unit 31. Thecorrection amount storing unit 31 stores a latest correction amount.Correction amounts recorded are, an exposure correction amount, a whitebalance correction amount, a contrast correction amount, a saturationcorrection amount, a sharpness correction amount and the like. Contentsto be stored are not limited to those mentioned here but vary withcorrection processing to be actually executed.

The data processing device 2 includes a correction amount obtaining unit21, a correction amount storing unit 22 and an image correcting unit 23.The correction amount obtaining unit 21 calculates a correction amountfrom a frame image obtained from the image input unit 1. The correctionamount storing unit 22 stores the correction amount obtained by thecorrection amount obtaining unit 21 at the correction amount storingunit 31.

The image correcting unit 23 subjects the frame image obtained from theimage input unit 1 to quality improving correction using the correctionamount stored in the correction amount storing unit 31 to output thecorrected image to the image output unit 4.

FIG. 2 is a block diagram showing a detailed structure of the correctionamount obtaining unit 21 illustrated in FIG. 1. In FIG. 2, thecorrection amount obtaining unit 21 includes an evaluation regioncut-out unit 211, a correction amount calculating unit 212, an upperlimit value adjusting unit 213, an upper limit value storing unit 214and a set value storing unit 215.

The evaluation region cut-out unit 211 cuts out an evaluation imageregion for use in calculating a correction amount from a frame image.The upper limit value storing unit 214 stores a maximum value that eachcorrection amount can take. The set value storing unit 215 stores adefault value of each correction amount.

The correction amount calculating unit 212 includes a saturationcorrection amount calculating unit 2121, a white balance correctionamount calculating unit 2122, a contrast correction amount calculatingunit 2123, an exposure correction amount calculating unit 2124, asharpness correction amount calculating unit 2125 and a preferable colorcorrection amount calculating unit 2126. Here, the correction amountcalculating unit 212 may be structured without one or more of thesecorrection amount calculating units. Also, these cited correction amountcalculating units are examples only and other correction amountcalculating unit may be incorporated. In FIG. 2, although the respectivecorrection amount calculating units are illustrated in parallel to eachother, they may sequentially execute their operation in an arbitraryorder.

The saturation correction amount calculating unit 2121 extracts afeature amount from an image for evaluation which is cut out by theevaluation region cut-out unit 211 to determine a saturation correctionamount based on the feature amount. The white balance correction amountcalculating unit 2122 extracts a feature amount from an image forevaluation cut out by the evaluation region cut-out unit 211 todetermine a white balance correction amount based on the feature amount.

The contrast correction amount calculating unit 2123 extracts a featureamount from an image for evaluation cut out by the evaluation regioncut-out unit 211 to determine a contrast correction amount based on thefeature amount. The exposure correction amount calculating unit 2124extracts a feature amount from an image for evaluation cut out by theevaluation region cut-out unit 211 to determine an exposure correctionamount based on the feature amount.

The sharpness correction amount calculating unit 2125 extracts a featureamount from an image for evaluation cut out by the evaluation regioncut-out unit 211 to determine a sharpness correction amount based on thefeature amount. The preferable color correction amount calculating unit2126 extracts a feature amount from an image for evaluation cut out bythe evaluation region cut-out unit 211 to determine a preferable colorcorrection amount for the correction into a preset preferable colorbased on the feature amount.

Here, preferable color correction is intended to realize color that onefinds preferable when looking only at a corrected image and is conductedbased on the contents of know-how accumulated in a data base for a longperiod of time. More specifically, correction is conducted such thathues related to skin color, color of the sky and green of plants becomehues which produce more preferable colors by adapting a color correctionparameter given in advance according to each divisional hue region. As aresult, more preferable colors are obtained with only the colors ofskin, the sky and green of plants changed. As to the above-describedpreferable color correction, recitation is found in Literature 4.

When any of correction amounts obtained by the correction amountcalculating unit 212 exceeds an upper limit value recorded in the upperlimit value storing unit 214, the upper limit value adjusting unit 213converts the amount into a set value stored in the set value storingunit 215.

FIG. 3 is a diagram for use in explaining a method of adjusting acorrection amount by the correction amount obtaining unit 21 using anupper limit value and a set value. In FIG. 3, when any of the correctionamounts obtained by the correction amount calculating unit 212 exceedsthe upper limit value recorded in the upper limit value storing unit214, the upper limit value adjusting unit 213 converts the amount intothe set value recorded in the set value storing unit 215.

FIG. 4 is a block diagram showing an example of a detailed structure ofthe image correcting unit 23 illustrated in FIG. 1. In FIG. 4, the imagecorrecting unit 23 includes a white balance correcting unit 231, acontrast correcting unit 232, an exposure correcting unit 233, asaturation correcting unit 234, a sharpness correcting unit 235 and apreferable color correcting unit 236. The order how these correctingunits are arranged is not limited to that illustrated in FIG. 4. Theunit 23 may lack in one or more of these correcting units or may furtherinclude other correcting unit.

The white balance correcting unit 231 subjects an input frame image towhite balance correction based on a white balance correction amountamong the correction amounts recorded in the correction amount storingunit 31. The contrast correcting unit 232 subjects an input frame imageto contrast correction based on a contrast correction amount among thecorrection amounts recorded in the correction amount storing unit 31.

The exposure correcting unit 233 subjects an input frame image toexposure correction based on an exposure correction amount among thecorrection amounts recorded in the correction amount storing unit 31.The saturation correcting unit 234 subjects an input frame image tosaturation correction based on a saturation correction amount among thecorrection amounts recorded in the correction amount storing unit 31.

The sharpness correcting unit 235 subjects an input frame image tosharpness correction based on a sharpness correction amount among thecorrection amounts recorded in the correction amount storing unit 31.The preferable color correcting unit 236 subjects an input frame imageto preferable color correction based on a preferable color correctionamount among the correction amounts recorded in the correction amountstoring unit 31.

FIG. 5 is a flow chart showing operation of the video processing deviceaccording to the first embodiment of the present invention, while FIGS.6 and 7 are flow charts showing operation of the correction amountobtaining unit 21 of FIG. 1. With reference to FIGS. 1 to 7, descriptionwill be made of the video processing device according to the firstembodiment of the present invention.

When the processing is started, the video processing device firstinitializes a storage region, a variable and the like (Step S1 in FIG.5). Thereafter, the video processing device obtains an image to becorrected (Step S2 in FIG. 5), calculates an image correction amountbased on the obtained image (Step S3 in FIG. 5) and stores thecalculated image correction amount (Step S4 in FIG. 5).

The video processing device subjects the input image to image correctionprocessing based on the calculated correction amount (Step S5 in FIG. 5)to output the corrected image (Step S6 in FIG. 5). Subsequently, thevideo processing device determines whether an image is applied or not(Step S7 in FIG. 5) and when it is applied, returns to Step S2 to obtainthe image and repeat the same processing as that described above. Whenno image is applied, the video processing device ends the processing.

Upon start of the processing, the correction amount obtaining unit 21cuts out an evaluation region for obtaining a correction amount from theframe image (Step S11 in FIG. 6) and calculates a saturation correctionamount for the cut out image for evaluation (Step S12 in FIG. 6).

Subsequent to the above-described processing, the correction amountobtaining unit 21 sequentially calculates an exposure correction amount(Step S13 in FIG. 6), a white balance correction amount (Step S14 inFIG. 6), a contrast correction amount (Step S15 in FIG. 6), a sharpnesscorrection amount (Step S 16 in FIG. 6) and a preferable colorcorrection amount (Step S17 in FIG. 6). The order of obtaining therespective correction amounts is not limited thereto. In addition, oneor more of the above-described correction amounts can be omitted orother correction amount may be calculated.

Next, the correction amount obtaining unit 21 confirms an upper limitvalue of the obtained correction amount. First, the correction amountobtaining unit 21 checks whether the saturation correction amountexceeds an upper limit value (Step S18 in FIG. 6) and when it exceedsthe upper limit value, sets the obtained saturation correction amount atthe set value (Step S19 in FIG. 6) and unless it exceeds the upper limitvalue, the unit uses the previously obtained saturation correctionamount.

Similarly, the correction amount obtaining unit 21 checks whether theexposure correction amount exceeds an upper limit value (Step S20 inFIG. 6) and when it exceeds the upper limit value, sets the obtainedexposure correction amount at a set value (Step S21 in FIG. 6) andunless it exceeds the upper limit value, the unit uses the previouslyobtained exposure correction amount.

The correction amount obtaining unit 21 checks whether the white balancecorrection amount exceeds an upper limit value (Step S22 in FIG. 7) andwhen it exceeds the upper limit value, sets the obtained white balancecorrection amount at a set value (Step S23 in FIG. 7) and unless itexceeds the upper limit value, the unit uses the previously obtainedwhite balance correction amount.

The correction amount obtaining unit 21 checks whether the contrastcorrection amount exceeds an upper limit value (Step S24 in FIG. 7) andwhen it exceeds the upper limit value, sets the obtained contrastcorrection amount at a set value (Step S25 in FIG. 7) and unless itexceeds the upper limit value, the unit uses the previously obtainedcontrast correction amount.

The correction amount obtaining unit 21 checks whether the sharpnesscorrection amount exceeds an upper limit value (Step S26 in FIG. 7) andwhen it exceeds the upper limit value, sets the obtained sharpnesscorrection amount at a set value (Step S27 in FIG. 7) and unless itexceeds the upper limit value, the unit uses the previously obtainedsharpness correction amount.

Lastly, the correction amount obtaining unit 21 checks whether thepreferable color correction amount exceeds an upper limit value (StepS28 in FIG. 7) and when it exceeds the upper limit value, sets theobtained preferable color correction amount at a set value (Step S29 inFIG. 7) and unless it exceeds the upper limit value, the unit uses thepreviously obtained preferable color correction amount. After executingthe foregoing steps, the correction amount obtaining unit 21 ends theprocessing.

At the time of determining a correction amount of a latest frame, bythus obtaining an amount of change between the correction amount of thelatest frame and that of a past frame to minimize the amount of changeto a degree that causes no flickering etc., quality of a moving imagecan be automatically improved without exhibiting uncomfortable look suchas flickering.

Moreover, because when the correction amount obtaining unit 21 evaluatesan image, an evaluation region can be cut out to have an arbitrary sizeby the evaluation region cut-out unit 211, the present embodimentenables automatic quality improving irrespective of a video source.

Furthermore, since the video processing device according to the presentembodiment is allowed to include the saturation correcting unit 234, theexposure correcting unit 233, the white balance correcting unit 231, thecontrast correcting unit 232, the sharpness correcting unit 235 and thepreferable color correcting unit 236 which are still picture automaticquality improving techniques in various combinations independently ofeach other, various kinds of high quality improving corrections can beautomatically conducted. In addition, not only the above-described sixunits but also other high quality improving unit can be incorporated toimprove quality of a moving image in the same manner as theabove-described six correcting units do.

FIG. 8 is a block diagram showing a structure of a video processingdevice according to a second embodiment of the present invention. InFIG. 8, the video processing device according to the second embodimentof the present invention has the same structure as that of the firstembodiment of the present invention shown in FIG. 1 with the onlydifference being that the data processing device 5 is provided with acorrection amount change limiting unit 24, in which the same componentsare indicated by the same reference numerals. Operation of the samecomponents is also identical to that in the first embodiment.

The correction amount change limiting unit 24 compares a correctionamount obtained by the correction amount obtaining unit 21 with acorrection amount of a preceding frame to change the correction amountaccording to the comparison result so as not to make a change amountexceed a fixed value.

FIG. 9 is a block diagram showing a detailed structure of the correctionamount change limiting unit 24 illustrated in FIG. 8. In FIG. 9, thecorrection amount change limiting unit 24 includes a change amountcalculating unit 241, a change amount limiting unit 242 and a maximumchange range storing unit 243.

The maximum change range storing unit 243 stores a maximum amount ofallowable change from a correction amount of a preceding frame to acorrection amount of a current frame in successive frame images.

The change amount calculating unit 241 obtains an absolute value of adifference between a correction amount of a latest frame obtained by thecorrection amount obtaining unit 21 and a correction amount of apreceding frame recorded in the correction amount storing unit 31 toacquire an amount of change in correction amount.

The change amount limiting unit 242 limits a correction amount of acurrent frame such that an amount of change in correction amountcalculated by the change amount calculating unit 241 will not exceed amaximum change range recorded in the maximum change range storing unit243.

FIG. 10 is a diagram showing one example of processing of the correctionamount change limiting unit 24 illustrated in FIG. 8. As illustrated inFIG. 10, the correction amount change limiting unit 24 obtains anabsolute value of a difference between a new correction amount and anold correction amount and when the value fails to exceed a maximumchange range, outputs the correction amount of the current frame withoutmodification.

If the absolute value of the difference exceeds the maximum changerange, the change correction amount change limiting unit 24 limits thecorrection amount of the current frame by making the change amount meetthe maximum change range such that the absolute value of the differencefalls within the change range and outputs the limited correction amount.

FIG. 11 is a flow chart showing operation of the video processing deviceaccording to the second embodiment of the present invention. Withreference to FIGS. 8 to 11, description will be made of operation of thevideo processing device according to the second embodiment of thepresent invention. In FIG. 11, since processing operation at Steps S31to S33 and S35 to S38 is the same as that of Steps S1 to S7 shown inFIG. 5, no description will be made thereof.

In the first embodiment of the present invention, when a correctionamount is obtained from an input image, the correction amount isrecorded without modification to subject the input image to imagecorrection. On the other hand, in the present embodiment, after acorrection amount is obtained (Step S33 in FIG. 11), an amount of changefrom a correction amount of a preceding frame when the correction amountis used without modification in image correction is limited within afixed value (Step S34 in FIG. 11).

As described above, in the present embodiment, a correction amount whoseamount of change in time is limited is recorded (Step S35 in FIG. 11)and based on the recorded amount, image correction is conducted withrespect to the input image (Step S36 in FIG. 11).

As described in the foregoing, by suppressing an amount of change intime of a correction amount within a range in which no flickering isperceived, the present embodiment eliminates a phenomenon of unnaturallook such as flickering. In other words, automatic improving of qualityof moving images is realized by changing a correction amount by thecorrection amount change limiting unit 24 within a range where noflickering is perceived.

According to the conventional methods, applying the still picturequality improving techniques to each frame image of a moving imageresults in that a correction amount varies with each frame because imagequality of each frame image slightly differs from each other. When anamount of correction changes largely in frame images adjacent to eachother in time, look of the image changes instantly, so that flickeringis perceived in a corrected moving image. The present embodiment solvesthe present problem.

FIG. 12 is a block diagram showing a structure of a video processingdevice according to a third embodiment of the present invention. In FIG.12, the video processing device according to the third embodiment of thepresent invention has the same structure as that of the secondembodiment of the present invention shown in FIG. 8 with the differencebeing that the data processing device 6 includes a correction amountupdate determining unit 25 and a change amount limit executiondetermining unit 26 and the storage device 7 includes a number of framesstoring unit 32, in which the same components are indicated by the samereference numerals. In addition, operation of the counterpart componentsis the same as that of the second embodiment.

Upon obtaining an image from the image input unit 1, the correctionamount update determining unit 25 increments the value of the number offrames storing unit 32 by one and detects a cut point from the obtainedframe image or when detecting the value of the number of frames storingunit 32 exceeding a fixed value, determines to update the correctionamount. In other words, the correction amount update determining unit 25generates a cut point detection signal when detecting a cut point andgenerates a lapse of fixed time signal when detecting a fixed timeelapsing.

The change amount limit execution determining unit 26 determines whetherto execute the correction amount change limiting unit 24 by a signalreceived from the correction amount update determining unit 25. Morespecifically, upon receiving the lapse of fixed time signal from thecorrection amount update determining unit 25, the change amount limitexecution determining unit 26 sends the correction amount obtained fromthe correction amount obtaining unit 21 to the correction amount changelimiting unit 24 and upon receiving the cut point detection signal fromthe correction amount update determining unit 25, sends the correctionamount to the correction amount storing unit 22.

FIG. 13 is a block diagram showing a detailed structure of thecorrection amount update determining unit 25 illustrated in FIG. 12. InFIG. 13, the correction amount update determining unit 25 includes aframe counting unit 251, a cut point detecting unit 252 and a lapse offixed time detecting unit 253.

When a frame is switched in an input image, the frame counting unit 251increments the number of frames stored in the number of frames storingunit 32 by one. The cut point detecting unit 252 extracts a featureamount from the input image and compares the feature amount with afeature amount extracted from a preceding frame to detect a cut point.Upon detection of a cut point, the frame counting unit 251 outputs thecut point detection signal to reset the number of frames storing unit32.

The lapse of fixed time detecting unit 253 checks the number of framesstored in the number of frames storing unit 32 to find whether thenumber exceeds a fixed value. Upon detecting the fixed time elapsing,the lapse of fixed time detecting unit 253 outputs the lapse of fixedtime signal to reset the number of frames storing unit 32.

FIG. 14 is a block diagram showing a detailed structure of the cut pointdetecting unit 252 of FIG. 13. In FIG. 14, the cut point detecting unit252 includes an evaluation region cut-out unit 2521, an image thinningout unit 2522, a histogram generating unit 2523, a histogram comparingunit 2524 and a histogram storing unit 2525.

FIG. 15 is a diagram for use in explaining a color histogram for use inthe cut point detecting unit 252 of FIG. 13, while FIG. 16 is a diagramshowing a relationship between transition of a difference value and athreshold value in the example of comparison in feature points used inthe cut point detecting unit 252 of FIG. 13. With reference to FIGS. 14to 16, operation of the cut point detecting unit 252 will be described.

The evaluation region cut-out unit 2521 cuts out an image region for usein cut point detection from an input frame image. The image thinning outunit 2522 extracts every n pixels (n≧1) from the image cut out by theevaluation region cut-out unit 2521 and combines the extracted pixels togenerate a thinned out image.

The histogram generating unit 2523 generates a color histogram based oncolor information of each pixel of the input frame image. Colorhistogram is, as illustrated in FIG. 15, a histogram generatedindependently for each of the RGB values as color information of eachpixel of the frame image.

The histogram storing unit 2525 stores a histogram extracted from apreceding frame. The histogram comparing unit 2524 compares the colorhistogram generated by the histogram generating unit 2523 and the colorhistogram of the preceding frame stored in the histogram storing unit2525 to determine whether a cut point exists between the frames based onthe obtained feature amounts.

Although used here as histogram comparison processing conducted by thehistogram comparing unit 2524 are techniques using a difference valueand a correlation value of a histogram and the like, the techniques arenot specifically limited. Description will be here made of a case wherea difference value of a histogram is used.

With a difference value of a histogram used, as a frame changes, itsdifference value changes as shown in FIG. 16. Between frames where a cutpoint exists, its difference value is apt to be larger than otherpoints. Therefore, as illustrated in FIG. 16, by providing a thresholdvalue to classify the points into a cut point when a difference value islarger than the threshold value and into a non-cut point when the sameis smaller, cut point detection is enabled. This cut point detectionmethod is applicable not only to such video processing as in the presentembodiment but also to compression of images, generation of a digestfrom video and the like.

FIG. 17 is a flow chart showing operation of the video processing deviceaccording the third embodiment of the present invention. With referenceto FIGS. 12, 13 and 17, description will be made of the video processingdevice according to the third embodiment of the present invention. Sinceprocessing operation at Steps S41, S42, S46, S48, S49 and S50 to S52 isthe same as that at Steps S31 to S38 in FIG. 11, no description will bemade thereof.

In the second embodiment of the present invention, when a correctionamount is obtained from an input image, a range of an amount ofallowable change between the obtained correction amount and a correctionamount obtained from a preceding frame is limited. On the other hand, inthe present embodiment, first add one to the frame counting unit 251every time a new frame image is input (Step S43 in FIG. 17).

In the present embodiment, when detecting a cut point in an input imageor when the count of the frame counting unit 251 exceeds a fixed value,determination is made to update a correction amount (Step S44 in FIG.17). When the correction amount will not be updated, image correction isconducted using a current correction amount (Step S50 in FIG. 17).

When the correction is to be updated, reset the frame counting unit 251to 0 (Step S45 in FIG. 17) to obtain a correction amount from thecurrent frame image (Step S46 in FIG. 17). At this time, when a cutpoint has been detected (Step S47 in FIG. 17), record the correctionamount here (Step S49 in FIG. 17) to conduct image correction. When nocut point has been detected (Step S47 in FIG. 17), limit the amount ofchange in a correction amount (Step S48 in FIG. 17) and record thecorrection amount whose change amount is limited (Step S49 in FIG. 17)to conduct image correction.

FIG. 18 is a flow chart showing operation of the correction amountupdate determining unit 25 illustrated in FIG. 12. With reference toFIGS. 12, 13 and 18, description will be made of operation of thecorrection amount update determining unit 25.

Upon start of the processing, the correction amount update determiningunit 25 increments the frame counting unit 251 by one (Step S61 in FIG.18) to check whether a cut point exists or not based on the frame image(Step S62 in FIG. 18).

The correction amount update determining unit 25 checks whether a cutpoint is detected or not (Step S63 in FIG. 18) and if it is detected,outputs the cut point detection signal (Step S64 in FIG. 18) to resetthe frame counting unit 251 to 0 (Step S67 in FIG. 18) and shifts theprocessing to the correction amount obtaining unit 21.

When no cut point is detected, the correction amount update determiningunit 25 checks whether the count of the frame counting unit 251 exceedsa fixed value or not (Step S65 in FIG. 18) and if it exceeds the fixedvalue, outputs a fixed time detection signal (Step S66 in FIG. 18) toset the frame counting unit 251 to 0 (Step S67 in FIG. 18) and shiftsthe processing to the correction amount obtaining unit 21. Upondetermining that the count exceeds the fixed value, the correctionamount update determining unit 25 shifts the processing directly to theimage correcting unit 23.

FIG. 19 is a flow chart showing operation of the cut point detectingunit 252 illustrated in FIG. 13. With reference to FIGS. 13, 14 and 19,description will be made of operation of the cut point detecting unit252.

Upon start of the processing, the cut point detecting unit 252 cuts outan image region for use in detecting a cut point from a frame image(Step S71 in FIG. 19), extracts every n pixels (n≧0) from the cut-outimage and combines the extracted pixels to form a thinned out image(Step S72 in FIG. 19).

The cut point detecting unit 252 generates a histogram based on thethinned-out image (Step S73 in FIG. 19) and compares the histogram witha histogram of a preceding frame (Step S74 in FIG. 19).

Based on the result of the comparison, the cut point detecting unit 252checks whether a cut point is detected or not (Step S75 in FIG. 19), andwhen it is detected, outputs the cut point detection signal (Step S76 inFIG. 19) and records the histogram generated by the present processing(Step S77 in FIG. 19) to end the processing.

In the present embodiment, since detection of a cut point enablesdetection of switching of a scene, quality improving can be conductedwith an appropriate correction amount according to different scenes.

When a scene in a moving image is switched, an image whose look differsfrom that of preceding images is applied to the system. When look of animage thus changes largely, a correction amount appropriate for eachimage might change in some cases.

With conventional methods, correction of a moving image with anappropriate correction amount is impossible because they use acorrection amount as a fixed parameter. On the other hand, according tothe present invention, when switching of a scene is detected, a newappropriate correction amount is automatically obtained by thecorrection amount obtaining unit 21. Therefore, quality improving can beconducted with an appropriate correction amount according to eachdifferent scene detected.

Moving image has a display region largely differing with its inputsource. In TV images and game images, images are displayed in the entirearea of a TV monitor. On the other hand, in hi-vision images, films,etc., black zones are displayed at upper and lower regions to have asmaller image display region.

In the present embodiment, there occurs a case where due to the blackzone, an appropriate correction amount can not be obtained. In contrastcorrection, for example, a correction amount is determined based on adark region of a screen. When the whole of the screen is used as anevaluation region, a correction amount will be determined based on theregion of upper and lower black zones to disable appropriate improvingof quality of an image at other region than the black zones.

Similarly in cut point detection, a cut point might not be detectedappropriately due to the effect of the black zones in some cases. It is,however, possible by the use of the here described evaluation regioncut-out unit 211 to solve these problems and to appropriately cut ascene to improve quality of a moving image with an appropriatecorrection amount.

Since according to the present invention, thinned-out images aregenerated at the time of cut point detection, a cut point can bedetected irrespective of interlace characteristics. In the presentembodiment, it is premised that various kinds of images such as a TVimage and a DVD image exist as an input image. Among these images, twoimages are seen overlapped with each other in one frame in some cases.This is a phenomenon occurring when a video signal as an interlace imagehas 30 frames/sec, while the main moving image has 24 frames/sec. Thedifference in frame rates causes such a phenomenon that two images areseen overlapped with each other in one frame.

When two images thus overlap with each other as described above, animage of a preceding scene and an image of a succeeding scene are seenoverlapped with each other in one frame at a cut point. This results inincreasing similarity between the preceding and the succeeding frames atthe cut point to hinder the above-described cut point detection in somecases. Therefore, thinning out images to eliminate overlap of imagesenables cut point detection to be conducted more satisfactorily.

FIG. 20 is a block diagram showing a structure of a video processingdevice according to a fourth embodiment of the present invention. InFIG. 20, the video processing device according to the fourth embodimentof the present invention has the same structure as that of the firstembodiment of the present invention shown in FIG. 1 with the onlydifference being that the data processing device 8 is provided with acorrection region cut-out unit 27 and an image composing unit 28, inwhich the same components are indicated by the same reference numerals.Operation of the counterpart components is the same as that of the firstembodiment of the present invention.

FIG. 21 is a schematic diagram showing processing of the videoprocessing device according to the fourth embodiment of the presentinvention. With reference to FIG. 21, the correction region cut-out unit27 and the image composing unit 28 will be described.

The correction region cut-out unit 27, as illustrated in FIG. 21( a),cuts out a region to be corrected from an image in which a moving imagepartly flows into such a cut-out image as shown in FIG. 21( b) and suchan image of a region not to be corrected as shown in FIG. 21( e). Thecut-out image shown in FIG. 21( b) will have its quality improved asillustrated in FIG. 21( c) by the quality improving technique describedin the first embodiment of the present invention.

The image composing unit 28 combines the image of a region not to becorrected which is cut out by the correction region cut-out unit 27 andthe image being corrected to generate such an input image as shown inFIG. 21( d). Here, the generated image is output from the image outputunit 4 to end the processing.

FIG. 22 is a flow chart showing operation of the video processing deviceaccording to the fourth embodiment of the present invention. Withreference to FIGS. 20 to 22, operation of the video processing deviceaccording to the fourth embodiment of the present invention will bedescribed. Since processing operation at Steps S81, S82, S84 to 86, S88and S89 is the same as that of the processing operation at Steps S1 toS7 of FIG. 5, no description will be made thereof.

In the first embodiment of the present invention, the whole of an inputimage is corrected as a region to be corrected. On the other hand, inthe present embodiment, first cut out a region where a moving image isdisplayed from an input image (Step S83 in FIG. 22).

In the present embodiment, obtain a correction amount for the cut outimage (Step S84 in FIG. 22) to subject only the cut out image tocorrection processing based on the obtained correction amount (Step S86in FIG. 22). The corrected image is combined with a region not to becorrected which is an image left after the region to be corrected is cutout (Step S87 in FIG. 22) to form the same screen as that of theoriginal image. In the present embodiment, the present processing willbe repeated until no further image is input.

Because at the time of executing quality improving correctionprocessing, the correction region cut-out unit 27 is allowed to cut outa moving image region from an input image and also because the imagecomposing unit 28 is provided to restore a cut-out moving image which isfurther subjected to quality improving correction to such arrangement asshown in an original computer screen, the present invention enablesquality improving and display of only a moving image region of an imagein which a moving image partly flows such as a computer screen.

Upon activation of such an application for displaying a moving image asa media player on a computer screen, an image appears in which stillpicture forms a surrounding area and a moving image flows locally. Withrespect to such an image, the correction region cut-out unit 27 cuts theimage into a region of still picture and a region of a moving image. Asa result, quality improve correction can be conducted with respect tothe moving image region using an appropriate correction amount. Thus,the corrected moving image is combined with the still picture region atthe surroundings by the image composing unit 28 to obtain a moving imagewhose quality is improved by a correction amount appropriate for amoving image, while maintaining such display as that on an originalcomputer screen.

FIG. 23 is a block diagram showing a structure of a video processingdevice according to a fifth embodiment of the present invention. In FIG.23, the video processing device according to the fifth embodiment of thepresent invention has the same structure as that of the first embodimentof the present invention shown in FIG. 1 with the only difference beingthat in the data processing device 9, an image corrected by an imagecorrecting unit 30 is returned to a correction amount obtaining unit 29,in which the same components are indicated by the same referencenumerals. In addition, operation of the counterpart components is thesame as that of the first embodiment of the present invention.

FIG. 24 is a block diagram showing a detailed structure of thecorrection amount obtaining unit 29 illustrated in FIG. 23. In FIG. 24,the correction amount obtaining unit 29 has the same structure as thatof the correction amount obtaining unit 21 shown in FIG. 2 with the onlydifference being that it is designed to input an image corrected by theimage correcting unit 30 to the evaluation region cut-out unit 211, inwhich the same components are indicated by the same reference numerals.In addition, operation of the counterpart components is the same as thatof the correction amount obtaining unit 21.

FIG. 25 is a block diagram showing an example of a detailed structure ofthe image correcting unit 30 illustrated in FIG. 23. In FIG. 25, theimage correcting unit 30 has the same structure as that of the imagecorrecting unit 23 shown in FIG. 4 with the only difference being that acorrected image buffer 237 is provided, in which the same components areindicated by the same reference numerals. In addition, operation of thecounterpart components is the same as that of the image correcting unit23.

With reference to FIGS. 23 to 25, description will be made ofcharacteristic operation of the video processing device according to thefifth embodiment of the present invention. Being structured to return animage corrected by the image correcting unit 30 to the correction amountobtaining unit 29, the video processing device according to the fifthembodiment of the present invention corrects an image by each correctionunit of the image correcting unit 30 based on a correction amountobtained by each calculation unit of the correction amount obtainingunit 29 and obtains a correction amount from the corrected image by eachcalculation unit of the correction amount obtaining unit 29.

More specifically, first, the saturation correcting unit 234 corrects aninput image based on a correction amount obtained by the saturationcorrection amount calculating unit 2121 and temporarily accumulates thecorrected image in the corrected image buffer 237, as well as returningthe corrected image to the correction amount obtaining unit 29.

Subsequently, the white balance correction amount calculating unit 2122obtains a correction amount from the corrected image obtained by thesaturation correcting unit 234. The white balance correcting unit 231corrects the corrected image which is obtained by the saturationcorrecting unit 234 and temporarily accumulated at the corrected imagebuffer 237 based on the correction amount obtained by the white balancecorrection amount calculating unit 2122 to temporarily accumulate thecorrected image at the corrected image buffer 237, as well as returningthe corrected image to the correction amount obtaining unit 29.

In the same manner as described above, the contrast correction amountcalculating unit 2123 obtains a correction amount from the correctedimage obtained by the white balance correcting unit 231. The contrastcorrecting unit 232 corrects the corrected image which is obtained bythe white balance correcting unit 231 and temporarily accumulated at thecorrected image buffer 237 based on the correction amount obtained bythe contrast correction amount calculating unit 2123 to temporarilyaccumulate the obtained corrected image in the corrected image buffer237, as well as returning the corrected image to the correction amountobtaining unit 29.

The exposure correction amount calculating unit 2124 obtains acorrection amount from the corrected image obtained by the contrastcorrecting unit 232. The exposure correcting unit 233 corrects thecorrected image which is obtained by the contrast correcting unit 232and temporarily accumulated at the corrected image buffer 237 based onthe correction amount obtained by the exposure correction amountcalculating unit 2124 to temporarily accumulate the obtained correctedimage in the corrected image buffer 237, as well as returning thecorrected image to the correction amount obtaining unit 29.

The sharpness correction amount calculating unit 2125 obtains acorrection amount from the corrected image obtained by the exposurecorrecting unit 233. The sharpness correcting unit 235 corrects thecorrected image which is obtained by the exposure correcting unit 233and temporarily accumulated at the corrected image buffer 237 based onthe correction amount obtained by the sharpness correction amountcalculating unit 2125 to temporarily accumulate the obtained correctedimage at the corrected image buffer 237, as well as returning thecorrected image to the correction amount obtaining unit 29.

The preferable color correction amount calculating unit 2126 obtains acorrection amount from the corrected image obtained by the sharpnesscorrecting unit 235. The preferable color correcting unit 236 correctsthe corrected image which is obtained by the sharpness correcting unit235 and temporarily accumulated at the corrected image buffer 237 basedon the correction amount obtained by the preferable color correctionamount calculating unit 2126 to temporarily accumulate the obtainedcorrected image at the corrected image buffer 237, as well as returningthe corrected image to the correction amount obtaining unit 29.

By thus conducting correction by each correction unit of the imagecorrecting unit 30 based on a correction amount obtained by eachcalculation unit of the correction amount obtaining unit 29 andobtaining a correction amount from the corrected image by means of eachcalculation unit of the correction amount obtaining unit 29, an inputimage can be appropriately corrected. The order of arrangement of theabove-described correction amount calculating units and the correctingunits is not limited to those shown in FIGS. 24 and 25. Any one or moreof the correction amount calculating units and the correcting units canbe omitted or other unit may be added. In this case, deletion andaddition of the above-described units are executed in a pair of thecorrection amount calculating unit and the correcting unit.

FIG. 26 is a block diagram showing a structure of a video display deviceaccording to a sixth embodiment of the present invention. In FIG. 26,the video display device according to the sixth embodiment of thepresent invention is a device using the above-described video processingdevice and composed of an image input unit 1 and an image displayapparatus 10.

The image input unit 1 is the same as that of the first embodiment ofthe present invention. The image display apparatus 10 includes a dataprocessing device 2, a storage device 3 and an image display unit(monitor) 11. Here, the data processing device 2 and the storage device3 are the same as those of the first embodiment of the presentinvention. The image display unit 11 is a CRT (cathode-ray tube)monitor, a liquid crystal monitor or the like.

When moving image data is applied through the image input unit 1,similarly to the processing by the first embodiment of the presentinvention, the data processing device 2 and the storage device 3 conductsuch correction as white balance correction, contrast correction,exposure correction, saturation correction, sharpness correction and thelike with respect to the moving image to improve the quality of theimage. Here, as to correction given to an image, not all the five kindsof corrections described here needs to be conducted and other correctionthan the corrections mentioned above, and such quality improvecorrection as preferable color correction may be added. Image subjectedto this correction to have its quality improved is displayed on theimage display unit 11.

FIG. 27 is a block diagram showing a structure of a video processingdevice according to a seventh embodiment of the present invention. InFIG. 27, the video processing device according to the seventh embodimentof the present invention includes a recording medium 15 which records aprogram of the above-described video processing method.

More specifically, the video processing device according to the seventhembodiment of the present invention includes a video input device 12 forreceiving input of a moving image, a personal computer (hereinafterreferred to as PC) 13 for processing a program, an image display device14 for displaying a processing result and the recording medium 15storing the program which is executed by the PC 13 and realizes theabove-described video processing method.

When a moving image is applied to the PC 13 through the video inputdevice 12, the PC 13 executes correction with respect to the movingimage based on the program of the video processing method forautomatically improving quality of a moving image which is recorded inthe recording medium 15. The corrected moving image is sent to the imagedisplay device 14 and displayed thereon.

As described in the foregoing, according to the present invention, atthe time of improving quality of sequentially applied video, byobtaining a correction amount from each frame image forming the inputmoving image in order to conduct quality improve correction of the inputmoving image and by subjecting the frame image to quality improvecorrection based on the obtained correction amount, a correction amountcan be changed appropriately according to quality of an input movingimage to automatically improve quality of the moving image.

Although the invention has been illustrated and described with respectto exemplary embodiment thereof, it should be understood by thoseskilled in the art that the foregoing and various other changes,omissions and additions may be made therein and thereto, withoutdeparting from the spirit and scope of the present invention. Therefore,the present invention should not be understood as limited to thespecific embodiment set out above but to include all possibleembodiments which can be embodies within a scope encompassed andequivalents thereof with respect to the feature set out in the appendedclaims.

1. A video processing device comprising: correction amount obtainingmeans for obtaining a correction amount from sequentially applied movingimages, image correcting means for subjecting an input moving image toquality improving correction processing based on a correction amountobtained by said correction amount obtaining means, image input meansfor obtaining a frame image from sequentially applied moving images andoutputting the image to said correction amount obtaining means, whereinsaid correction amount obtaining means obtains said correction amountfrom said frame image and said image correcting means subjects saidframe image to quality improving correction processing based on saidcorrection amount, lapse of fixed time detecting means for counting thenumber of frames from a frame image whose said correction amount islastly updated to a current frame image to determine whether the numberof frames exceeds a fixed value, and correction amount updatedetermining means for giving an instruction to update said correctionamount when said lapse of fixed time detecting means determines that afixed time has elapsed.
 2. A video processing device comprising:correction amount obtaining means for obtaining a correction amount fromsequentially applied moving images, image correcting means forsubjecting an input moving image to quality improving correctionprocessing based on a correction amount obtained by said correctionamount obtaining means, image input means for obtaining a frame imagefrom sequentially applied moving images and outputting the image to saidcorrection amount obtaining means, wherein said correction amountobtaining means obtains said correction amount from said frame image andsaid image correcting means subjects said frame image to qualityimproving correction processing based on said correction amount, lapseof fixed time detecting means for counting the number of frames from aframe image whose said correction amount is lastly updated to a currentframe image to determine whether the number of frames exceeds a fixedvalue, cut point detecting means for detecting a cut point indicative ofswitching of a scene in said moving image based on a change of a featureamount obtained from each frame image, and correction amount updatedetermining means for giving an instruction to update said correctionamount either when detection of a lapse of a fixed time is made by saidlapse of fixed time detecting means or when detection of said cut pointis made by said cut point detecting means.
 3. A video processing devicecomprising: correction amount obtaining means for obtaining a correctionamount from sequentially applied moving images, image correcting meansfor subjecting an input moving image to quality improving correctionprocessing based on a correction amount obtained by said correctionamount obtaining means, image input means for obtaining a frame imagefrom sequentially applied moving images and outputting the image to saidcorrection amount obtaining means, wherein said correction amountobtaining means obtains said correction amount from said frame image andsaid image correcting means subjects said frame image to qualityimproving correction processing based on said correction amount, and cutpoint detecting means for detecting a cut point indicative of switchingof a scene in said moving image based on a change of a feature amountobtained from each frame image, and correction amount update determiningmeans for giving an instruction to update said correction amount whensaid cut point detecting means detects said cut point, wherein said cutpoint detecting means is structured to consider a result of comparisonof a color histogram generated based on color information of each pixelof said moving image which is conducted on a frame basis as a featureamount and detect a cut point of said moving image based on a change ofthe feature amount and wherein said cut point detecting means isstructured to, at the time of generating said color histogram from saidmoving image, generate said color histogram after thinning out the imageat fixed intervals.
 4. A video processing device comprising: image inputmeans for obtaining a frame image from sequentially applied movingimages, and cut point detecting means for detecting a cut pointindicative of switching of a scene in said moving image based on achange of a feature amount obtained from each frame image, wherein saidcut point detecting means is structured to consider a result ofcomparison of a color histogram generated based on color information ofeach pixel of said moving image which is conducted on a frame basis as afeature amount and detect a cut point of said moving image based on achange of the feature amount, and wherein said cut point detecting meansis structured to, at the time of generating said color histogram fromsaid moving image, generate said color histogram after thinning out theimage at fixed intervals.
 5. A video processing method comprising thesteps of: obtaining a correction amount from sequentially applied movingimages, conducting quality improving correction with respect to saidapplied moving image based on the obtained correction amount, andchecking said input moving image on a frame basis and when a cut pointindicative of switching of a scene in said input moving image isdetected, updating a correction amount at the detection of said cutpoint considering a result of comparison of a color histogram generatedbased on color information of each pixel of said moving image which isconducted on a frame basis as a feature amount and detecting a cut pointof the moving image based on a change of the feature amount, and whendetecting said cut point, at the time of generating said color histogramfrom said frame image, generating said color histogram after thinningout the image at fixed intervals.
 6. A video processing methodcomprising the steps of: obtaining a frame image from sequentiallyapplied moving images, and detecting a cut point indicative of switchingof a scene in said moving image based on a change of a feature amountobtained from each frame image, wherein at said cut point detectingstep, a result of comparison of a color histogram generated based oncolor information of each pixel of said moving image which is conductedon a frame basis is considered as a feature amount and a cut point ofsaid moving image is detected based on a change of the feature amount,and wherein at said cut point detecting step, at the time of generatingsaid color histogram from said moving image, said color histogram isgenerated after thinning out the image at fixed intervals.
 7. A videoprocessing method comprising the steps of: a correction amount obtainingstep of obtaining a correction amount from sequentially applied movingimages, an image correcting step of subjecting an input moving image toquality improving correction processing based on a correction amountobtained by said correction amount obtaining step, an image input stepof obtaining said a frame image from sequentially applied moving imagesand outputting the image to said correction amount obtaining step,wherein said correction amount obtaining step obtains said correctionamount from said frame image and said image correcting step subjectssaid frame image to quality improving correction processing based onsaid correction amount, lapse of fixed time detecting step of countingthe number of frames from a frame image whose said correction amount islastly updated to a current frame image to determine whether the numberof frames exceeds a fixed value, and correction amount updatedetermining step of giving an instruction to update said correctionamount when said lapse of fixed time detecting step determines that afixed time has elapsed.
 8. A video processing method comprising thesteps of: correction amount obtaining step of obtaining a correctionamount from sequentially applied moving images, image correcting step ofsubjecting an input moving image to quality improving correctionprocessing based on a correction amount obtained by said correctionamount obtaining step, image input step of obtaining a frame image fromsequentially applied moving images and outputting the image to saidcorrection amount obtaining step, wherein said correction amountobtaining step obtains said correction amount from said frame image andsaid image correcting step subjects said frame image to qualityimproving correction processing based on said correction amount, lapseof fixed time detecting step of counting the number of frames from aframe image whose said correction amount is lastly updated to a currentframe image to determine whether the number of frames exceeds a fixedvalue, cut point detecting step of detecting a cut point indicative ofswitching of a scene in said moving image based on a change of a featureamount obtained from each frame image, and correction amount updatedetermining step of giving an instruction to update said correctionamount either when detection of a lapse of a fixed time is made by saidlapse of fixed time detecting step or when detection of said cut pointis made by said cut point detecting step.